FACS Deep Learning Tool

"FACS Deep Learning Tool" is an innovative project that I contributed to as a research and technical artist during my tenure at "Eugenia Digital Humans" in Mexico City. This project gained recognition as one of the winners in the Epic Games MegaGrant program in the Winter of 2021.

My role involved designing and creating the complete pipeline for the Deep Learning Real-Time Recognition System. As part of the rapid methodology applied to the project, I developed a prototype showcasing an Emotion Real-Time Recognition System capable of identifying seven basic emotions: Neutral, Happy, Angry, Fearful, Disgusted, Sadness, and Surprise.

The overarching goal of the project was to develop a deep learning-based real-time recognition system capable of identifying up to 65 custom expressions (originally 110 expressions). This surpassed the capabilities of existing systems such as Faceware (48 expressions) and ARKIT (52 expressions), while still aligning with the well-established FACS Action Units developed since the 1970s.

To train the model, we utilized public datasets such as CK+ and FER-2013, while also creating our own custom database. This database combined our own custom Action Units (AU's) with existing ones from FACS, ARKIT, FaceWare, and Melinda ozel.

The project leveraged several libraries, including TensorFlow, Keras, Python, OpenCV, and PyQt5, to enable seamless implementation and integration.